Skip to content

Experiment: use msgspec to parse Formio (component) definitions - #5912

Draft
sergei-maertens wants to merge 1 commit into
mainfrom
experiment/formio-components-as-python-types
Draft

Experiment: use msgspec to parse Formio (component) definitions#5912
sergei-maertens wants to merge 1 commit into
mainfrom
experiment/formio-components-as-python-types

Conversation

@sergei-maertens

@sergei-maertens sergei-maertens commented Jan 22, 2026

Copy link
Copy Markdown
Member

[skip: e2e]

Plan to get this in main

The impact is huge and big bang change is pretty much the only way to get this through, as it affects all the component-specific processing which happens in all layers of the application. Though, this PR will get split up to do this in a responsible way.

  1. Implement all the changes in this PR and get the test suite to still pass.
  2. Once that is done (or gradually), the fixes/improvements to test data setup can be extracted and merged into main. That's a net benefit either way.
  3. The Formio type definitions and msgspec-specific parsing code can be extracted and merged, without wiring things up. At this point, the type definitions are fully validated because of 1, and few changes will be expected in that area.
  4. Our own test factories can be updated to run the msgspec validation. This prevents further issues from seeping into (new) tests - relates to 2.
  5. Once the type conversion/parsing code is present in a release, we can define a check script that loads all existing form definitions on (SAAS) instances. This allows us to find gaps between the type definitions and the reality.
  6. With the findings from 6, we can write fix scripts/data migrations/converters that bring existing data properly in line.
  7. We can add a sanity check in the form definition serializer that validates that whatever the API submits, is understood and correctly shaped. This stops new issues seeping into data and relates to 5/6.
  8. Merge the refactor that performs the actual processing on msgspec structs & get that released.

Changes

[Describe the changes here]

Checklist

Check off the items that are completed or not relevant.

  • Impact on features

    • Checked copying a form
    • Checked import/export of a form
    • Config checks in the configuration overview admin page
    • Problem detection in the admin email digest is handled
  • Dockerfile/scripts

    • Updated the Dockerfile with the necessary scripts from the ./bin folder
  • Commit hygiene

    • Commit messages refer to the relevant Github issue
    • Commit messages explain the "why" of change, not the how

@sergei-maertens
sergei-maertens force-pushed the experiment/formio-components-as-python-types branch 4 times, most recently from 32ee4f6 to 4fa6ea6 Compare January 23, 2026 14:08
@codecov

codecov Bot commented Jan 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.89346% with 141 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.90%. Comparing base (f1541e1) to head (89ca615).

Files with missing lines Patch % Lines
src/openforms/formio/components/custom.py 92.69% 2 Missing and 11 partials ⚠️
src/openforms/formio/components/vanilla.py 95.33% 0 Missing and 12 partials ⚠️
src/formio_types/select.py 84.12% 5 Missing and 5 partials ⚠️
src/formio_types/datetime.py 88.46% 5 Missing and 4 partials ⚠️
src/openforms/formio/utils.py 78.04% 4 Missing and 5 partials ⚠️
src/openforms/formio/registry.py 89.47% 1 Missing and 7 partials ⚠️
src/openforms/formio/datastructures.py 85.41% 5 Missing and 2 partials ⚠️
src/formio_types/bsn.py 85.71% 3 Missing and 3 partials ⚠️
src/formio_types/postcode.py 85.71% 3 Missing and 3 partials ⚠️
src/formio_types/radio.py 88.46% 3 Missing and 3 partials ⚠️
... and 17 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5912      +/-   ##
==========================================
- Coverage   97.13%   96.90%   -0.23%     
==========================================
  Files         861      893      +32     
  Lines       33067    34583    +1516     
  Branches     3054     3167     +113     
==========================================
+ Hits        32120    33514    +1394     
- Misses        635      684      +49     
- Partials      312      385      +73     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@sergei-maertens
sergei-maertens force-pushed the experiment/formio-components-as-python-types branch 4 times, most recently from 9fd5294 to 8409a3a Compare January 25, 2026 21:34
@sergei-maertens
sergei-maertens force-pushed the experiment/formio-components-as-python-types branch from 8409a3a to df8b627 Compare April 10, 2026 06:37
@sergei-maertens
sergei-maertens force-pushed the experiment/formio-components-as-python-types branch 6 times, most recently from 76b1c8a to 9fae907 Compare May 25, 2026 18:37
@sergei-maertens
sergei-maertens force-pushed the experiment/formio-components-as-python-types branch 6 times, most recently from 4f3c956 to d3836e9 Compare May 31, 2026 18:04
@sergei-maertens
sergei-maertens force-pushed the experiment/formio-components-as-python-types branch 4 times, most recently from dad3943 to 8051a44 Compare June 14, 2026 14:20
@sergei-maertens
sergei-maertens force-pushed the experiment/formio-components-as-python-types branch 2 times, most recently from 8bc8715 to c60491f Compare June 14, 2026 20:05
@sergei-maertens
sergei-maertens force-pushed the experiment/formio-components-as-python-types branch 6 times, most recently from 626b2d3 to 88aaee8 Compare July 27, 2026 07:45
@sergei-maertens
sergei-maertens force-pushed the experiment/formio-components-as-python-types branch 8 times, most recently from ee75ab8 to febdec2 Compare August 2, 2026 21:53
@sergei-maertens
sergei-maertens force-pushed the experiment/formio-components-as-python-types branch 7 times, most recently from 5e783f7 to 6bb9ad5 Compare August 21, 2026 21:21
@sergei-maertens
sergei-maertens force-pushed the experiment/formio-components-as-python-types branch 4 times, most recently from d667502 to 2540f91 Compare August 24, 2026 21:24
@sergei-maertens
sergei-maertens force-pushed the experiment/formio-components-as-python-types branch 2 times, most recently from f936144 to b0dd733 Compare September 7, 2026 19:34
POC for a proper data types rewrite to deal with anything formio in the backend.

See #6565
@sergei-maertens
sergei-maertens force-pushed the experiment/formio-components-as-python-types branch from b0dd733 to 188f047 Compare September 8, 2026 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant